Lab 3 - Handling Form Submission with Express

COMP7270 - Web and Mobile Programming - HKBU - Spring2024

Last week, we created an HTML form, and today we will explore how the data collected by this form can be submitted to a remote server. Additionally, we will develop our own server implementation to handle the form submission.

Currently, when we click the "submit" button, no data is being submitted anywhere because we haven't defined the action attribute. The action attribute specifies the receiver of the form submission.

To begin, let's utilize a free online service before we proceed with building our own server. We can add the following attribute to the opening tag of the <form> element:

action="https://www.httpbin.org/post"
action="https://www.httpbin.org/post"

This particular server will echo back everything that is submitted to it.

HTTP Method - GET and POST

In addition to the action attribute, another crucial attribute for form submission is method, where we define the HTTP method to be used.

When working with HTML forms, the commonly used HTTP methods are GET and POST. Here's a rule of thumb to help you decide which method to use:

Since we are adding data to the server-side database in this case, we will use the POST method. Let's add the following attribute to the opening tag of the <form> element:

method="POST"
method="POST"

Now, click the submit button again. Here's the response from httpbin.org:

{
  "args": {},
  "data": "",
  "files": {},
  "form": {
    "gridRadios": "Credit Card"
  },
  "headers": {
    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
    "Accept-Encoding": "gzip, deflate, br",
    "Accept-Language": "en-GB,en;q=0.9",
    "Content-Length": "18",
    "Content-Type": "application/x-www-form-urlencoded",
    "Host": "www.httpbin.org",
    "Origin": "http://127.0.0.1:5500",
    "Referer": "http://127.0.0.1:5500/",
    "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.5 Safari/605.1.15",
    "X-Amzn-Trace-Id": "Root=1-62f89dc4-525ffe5d37d8398257ff2c7c"
  },
  "json": null,
  "origin": "158.182.194.232",
  "url": "https://www.httpbin.org/post"
}
{
  "args": {},
  "data": "",
  "files": {},
  "form": {
    "gridRadios": "Credit Card"
  },
  "headers": {
    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
    "Accept-Encoding": "gzip, deflate, br",
    "Accept-Language": "en-GB,en;q=0.9",
    "Content-Length": "18",
    "Content-Type": "application/x-www-form-urlencoded",
    "Host": "www.httpbin.org",
    "Origin": "http://127.0.0.1:5500",
    "Referer": "http://127.0.0.1:5500/",
    "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.5 Safari/605.1.15",
    "X-Amzn-Trace-Id": "Root=1-62f89dc4-525ffe5d37d8398257ff2c7c"
  },
  "json": null,
  "origin": "158.182.194.232",
  "url": "https://www.httpbin.org/post"
}

Although there is a lot of interesting information available in the headers section, we seem to be missing the form data. So, what have we missed?

The name Attribute

In an HTML form, the input fields are identified using the name attributes. Let's add the name attribute to each form element. For example:

<input class="input is-danger" type="email" placeholder="Email input" name="email">
<input class="input is-danger" type="email" placeholder="Email input" name="email">

Additionally, for radio buttons belonging to the same group, they should be assigned the same name attribute. Fill in the form and submit it again. Here's a sample output:

{
  "args": {}, 
  "data": "", 
  "files": {}, 
  "form": {
    "email": "tony@stark.com", 
    "numTickets": "2", 
    "payment": "Paypal", 
    "superhero": "Iron Man", 
    "team": "Avengers", 
    "terms": "on"
  }, 
  "headers": {
    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", 
    "Accept-Encoding": "gzip, deflate, br", 
    "Accept-Language": "en-GB,en;q=0.9", 
    "Content-Length": "92", 
    "Content-Type": "application/x-www-form-urlencoded", 
    "Host": "www.httpbin.org", 
    "Origin": "http://127.0.0.1:5500", 
    "Referer": "http://127.0.0.1:5500/", 
    "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.5 Safari/605.1.15", 
    "X-Amzn-Trace-Id": "Root=1-62f89eec-0882635642e565536d82987b"
  }, 
  "json": null, 
  "origin": "158.182.194.232", 
  "url": "https://www.httpbin.org/post"
}
{
  "args": {}, 
  "data": "", 
  "files": {}, 
  "form": {
    "email": "tony@stark.com", 
    "numTickets": "2", 
    "payment": "Paypal", 
    "superhero": "Iron Man", 
    "team": "Avengers", 
    "terms": "on"
  }, 
  "headers": {
    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8", 
    "Accept-Encoding": "gzip, deflate, br", 
    "Accept-Language": "en-GB,en;q=0.9", 
    "Content-Length": "92", 
    "Content-Type": "application/x-www-form-urlencoded", 
    "Host": "www.httpbin.org", 
    "Origin": "http://127.0.0.1:5500", 
    "Referer": "http://127.0.0.1:5500/", 
    "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.5 Safari/605.1.15", 
    "X-Amzn-Trace-Id": "Root=1-62f89eec-0882635642e565536d82987b"
  }, 
  "json": null, 
  "origin": "158.182.194.232", 
  "url": "https://www.httpbin.org/post"
}

The submitted form data is now available under the form property. Notice that when a checkbox element is not assigned a value attribute, the submitted value will be on when the box is checked. Now, let's guess what will be submitted when this box is unchecked?

Node.js

Now, our form is ready, and let's proceed with building our server-side script using Node.js. Node.js is a server-side JavaScript runtime environment that allows us to execute JavaScript code outside of the browser. In order to follow along, make sure you have Node.js installed on your machine. Please refer to the instructions provided in the appendix.

To start using Node.js, open your terminal or command prompt and type node to launch the Node.js environment.

JavaScript Variables and Data Types

JavaScript is a dynamically typed and loosely typed language. This means that variables in JavaScript are not bound to a specific value type. In JavaScript, you can assign and reassign values of different types to variables without explicitly declaring their type.

> var foo;
undefined

> foo = true
true
> typeof foo
'boolean'

> foo = 12
12
> typeof foo
'number'

> foo = "7Eleven"
'7Eleven'
> typeof foo
'string'
> var foo;
undefined

> foo = true
true
> typeof foo
'boolean'

> foo = 12
12
> typeof foo
'number'

> foo = "7Eleven"
'7Eleven'
> typeof foo
'string'

We can always use the typeof operator to determine the data type of a variable in JavaScript.

Type Conversion

The parseInt() function is used to parse a string and extract an integer value from it. Here are a couple of examples:

> parseInt("7Eleven")
7
> parseInt("Seven Eleven")
NaN
> parseInt("7Eleven")
7
> parseInt("Seven Eleven")
NaN

In the first example, the string "7Eleven" can be converted to an integer, and parseInt() returns the value 7. However, in the second example, the string "Seven Eleven" cannot be converted to a numeric value, and parseInt() returns NaN, which stands for "Not a Number".

To check if a string can be successfully parsed as a number, you can use the global function `isNaN()``:

> parseInt("Seven Eleven") == NaN
false
> parseInt("Seven Eleven") == "NaN"
false
> isNaN(parseInt("Seven Eleven"))
true
> parseInt("Seven Eleven") == NaN
false
> parseInt("Seven Eleven") == "NaN"
false
> isNaN(parseInt("Seven Eleven"))
true

JavaScript Object and Array

The object is an important data type in JavaScript that allows you to store and manipulate collections of key-value pairs:

> foo = { a: 1, b: 2 }
{ a: 1, b: 2 }
> typeof foo
'object'
> foo = { a: 1, b: 2 }
{ a: 1, b: 2 }
> typeof foo
'object'

To delete a property from an object, you can use the delete operator. You can also add a new property to an object using a regular assignment statement:

> delete foo.a
true
> foo.c = 3
3
> foo
{ b: 2, c: 3 }
> delete foo.a
true
> foo.c = 3
3
> foo
{ b: 2, c: 3 }

Additionally, it's worth noting that an array in JavaScript is also considered an object, albeit with hidden numeric keys:

> foo = [1, 2, 3]
[ 1, 2, 3 ]
> typeof foo
'object'
> foo = [1, 2, 3]
[ 1, 2, 3 ]
> typeof foo
'object'

The undefined Type

There is a special data type in JavaScript, the undefined type. Variables with undefined values will carry this undefined type:

In JavaScript, there is a special data type called undefined. Variables that have not been assigned a value are said to have an undefined value, and their type is undefined. Here's an example:

> var undefinedVar;
undefined
> typeof undefinedVar
'undefined'
> var undefinedVar;
undefined
> typeof undefinedVar
'undefined'

Truthy and Falsy

In JavaScript, values that are considered either truthy or falsy. The following values are always falsy:

Conversely, any other value not listed above is considered truthy.

You can use these truthy and falsy values within conditional statements. For example:

> if (parseInt("7Eleven")) foo = "It's truthy"
"It's truthy"

> if (parseInt("Eleven")) foo = "It's truthy"; else foo = "It's falsy"
"It's falsy"
> if (parseInt("7Eleven")) foo = "It's truthy"
"It's truthy"

> if (parseInt("Eleven")) foo = "It's truthy"; else foo = "It's falsy"
"It's falsy"

Understanding truthy and falsy values allows you to write more expressive and concise code when working with conditions in JavaScript.

Fallback Value

In JavaScript, you can use the logical OR operator (||) to assign a fallback value for a falsy value. The operator || allows you to specify a default value that will be used if the first operand is falsy. Here's an example:

> foo = undefinedVar || "fallback value"
'fallback value'
> foo = undefinedVar || "fallback value"
'fallback value'

Tenary Operator

The JavaScript ternary operator, also known as the conditional operator, provides a concise way to write conditional statements. It takes three operands and evaluates a condition, returning one of two values based on whether the condition is true or false.

> foo = foo? true: false
true
> foo = foo? true: false
true

Equality Comparisons

In JavaScript, there are two types of equality comparisons: loose (==) and strict (===).

Loose equality comparisons (==) perform type coercion, meaning they attempt to convert the operands to a common type before making the comparison. On the other hand, strict equality comparisons (===) do not perform type coercion and require both the value and the type to be the same for the comparison to be true.

Here's an example to illustrate the difference:

> "12" == 12
true
> "12" === 12
false
> "12" == 12
true
> "12" === 12
false

Express

Node.js is a versatile technology that can be used for various tasks, but its primary usage is in building web applications. To streamline and expedite the development process, developers often rely on the Express web application framework.

Express is a popular and lightweight framework for Node.js that provides a robust set of features and utilities to simplify web application development. It offers a minimalistic approach, allowing developers to create scalable and efficient web applications quickly.

enter image description here

By utilizing Express, developers can handle routing, implement middleware, manage HTTP requests and responses, and build APIs with ease. Express also has a vast ecosystem of plugins and extensions, making it highly customizable and adaptable to different project requirements.

To generate a new folder named lab03 with a pre-built file structure (scaffold) using Express, follow these steps in your terminal:

npx express-generator --view=ejs --git lab03
npx express-generator --view=ejs --git lab03

Running this command will automatically create the necessary files and folders for the project. The ejs parameter specifies the chosen template engine. Alternatively, you can choose other options like Jade or Handlebars for your project.

Once the command completes execution, you will find the generated folder lab03 with the skeleton files inside. This scaffold provides a starting point for your web application development using Express.

After navigating into the lab03 folder, you can proceed with installing the required dependencies by running the following command in your terminal:

cd lab03
npm install
cd lab03
npm install

This command fetches and installs all the necessary Node.js packages and modules listed in the package.json file. Please note that the installation process may take some time to complete.

During the installation, you might come across a critical severity vulnerability. To address this, you can run the following command:

npm audit fix --force
npm audit fix --force

Executing this command will attempt to fix the vulnerability by upgrading the ejs module from version 2 to version 3.

Additionally, please copy the index.html file to the public folder of the newly created project. This file will be served as a static file by the server.

Once you have copied the index.html file, you can start the server by running the following command:

npm start
npm start

By default, the server will run on port 3000. Therefore, you can access the website by visiting http://localhost:3000/ in your web browser. If you encounter any issues accessing the website, ensure that any proxy settings are turned off.

After accessing the website, you should see the same form that was developed earlier.

Echoing the From Inputs

To enable echoing of incoming data similar to httpbin.org, you can locate the file index.js inside the routes folder of your project, and register the following route handler:

/* Handle the Form */
router.post('/booking', async function (req, res) {

    var response = {
        headers: req.headers,
        body: req.body
    };

    res.json(response);    
});
/* Handle the Form */
router.post('/booking', async function (req, res) {

    var response = {
        headers: req.headers,
        body: req.body
    };

    res.json(response);    
});

This route handler will be triggered whenever a client visits the path /booking with the POST method. It will send back the headers and request body as a JSON response to the client.

The req and res arguments in the route handler function represent the HTTP request and response objects, respectively.

Modify the action attribute in the index.html file as follows:

action="/booking"
action="/booking"

This ensures that the form data will be submitted to our application at the /booking endpoint.

Stop your server by pressing Ctrl + C in your terminal. Restart the server by running the npm start command again. Submit the form on the website to the /booking endpoint.

Upon submitting the form, your application will send back a JSON response containing the headers and request body to the client.

If you find the JSON response difficult to read, you can add the following line to the app.js file, around line 15:

app.set('json spaces', 2);
app.set('json spaces', 2);

This will "prettify" the JSON output, making it more readable.

Saving Data

To save data in your web application, you have various options for connecting to different database engines such as MongoDB, MySQL, or Oracle. In this case, we will explore using Azure Cosmos DB, which is a cloud-based solution that offers an API for MongoDB. By utilizing this API, you can interact with Cosmos DB as if it were a MongoDB database.

To get started, please follow these steps:

Visit the following URL: https://azure.microsoft.com/en-gb/free/students/ (Azure for Students – Free Account Credit | Microsoft Azure).

Sign in using your HKBU @Life account, which should be in the format xxxxxxxx@life.hkbu.edu.hk. Once signed in, select Access student benefits." Fill in the required personal details, including your phone number.

After completing the above steps, you will receive $100 in stored credits, and there is no need to enter any credit card information.

After completing the registration process, you will be directed to the home page.

In the creation form, choose a new Resource Group or create one by clicking on Create new and providing a name for it. Enter a unique Account Name for your Cosmos DB instance. Location should be East Asia. Make sure to select the serverless option for the capacity mode.

The MongoDB Node.js Driver

To connect to Azure Cosmos DB via the MongoDB API and store data on the server-side, follow these steps:

  1. In your terminal, run the following command to install the MongoDB Node.js Driver:

    npm install mongodb --save
    npm install mongodb --save

  2. Create a new folder named utils and within that folder, create a new file named db.js. Copy and paste the following code into the db.js file:

    const { MongoClient, ObjectId } = require('mongodb');

process.env.MONGODB_URI = 'mongodb://<username>:<password>@<endpoint>.documents.azure.com:10255/?ssl=true';

if (!process.env.MONGODB_URI) {
    // throw new Error('Please define the MONGODB_URI environment variable inside .env.local');
    process.env.MONGODB_URI = 'mongodb://localhost:27017';
}

// Connect to MongoDB
async function connectToDB() {
    const client = await MongoClient.connect(process.env.MONGODB_URI);
    const db = client.db('bookingsDB');
    db.client = client;
    return db;
}

module.exports = { connectToDB, ObjectId };
    const { MongoClient, ObjectId } = require('mongodb');

process.env.MONGODB_URI = 'mongodb://<username>:<password>@<endpoint>.documents.azure.com:10255/?ssl=true';

if (!process.env.MONGODB_URI) {
    // throw new Error('Please define the MONGODB_URI environment variable inside .env.local');
    process.env.MONGODB_URI = 'mongodb://localhost:27017';
}

// Connect to MongoDB
async function connectToDB() {
    const client = await MongoClient.connect(process.env.MONGODB_URI);
    const db = client.db('bookingsDB');
    db.client = client;
    return db;
}

module.exports = { connectToDB, ObjectId };

    Replace <username>, <password>, and <endpoint> in the process.env.MONGODB_URI string with the appropriate values from your Azure Cosmos DB connection string.

  3. In your index.js file, import the connectToDB and ObjectId functions by adding the following line near line 3 of the file:

    const { connectToDB, ObjectId } = require('../utils/db');
    const { connectToDB, ObjectId } = require('../utils/db');

    MongoDB has a special requirement for the primary key, which is named _id and has the data type ObjectId. Therefore, we need to import the ObjectId function as mentioned above.

  4. Implement the following code in the route handler for storing data in the server-side:

    /* Handle the Form */
router.post('/booking', async function (req, res) {
  const db = await connectToDB();
  try {
    req.body.numTickets = parseInt(req.body.numTickets);
    req.body.terms = req.body.terms? true : false;
    req.body.created_at = new Date();
    req.body.modified_at = new Date();

    let result = await db.collection("bookings").insertOne(req.body);
    res.status(201).json({ id: result.insertedId });
  } catch (err) {
    res.status(400).json({ message: err.message });
  } finally {
    await db.client.close();
  }
});
    /* Handle the Form */
router.post('/booking', async function (req, res) {
  const db = await connectToDB();
  try {
    req.body.numTickets = parseInt(req.body.numTickets);
    req.body.terms = req.body.terms? true : false;
    req.body.created_at = new Date();
    req.body.modified_at = new Date();

    let result = await db.collection("bookings").insertOne(req.body);
    res.status(201).json({ id: result.insertedId });
  } catch (err) {
    res.status(400).json({ message: err.message });
  } finally {
    await db.client.close();
  }
});

    This code snippet converts the numTickets property from a string to an integer using parseInt(), checks if the terms property is truthy and assigns the resulting Boolean value, sets the created_at and modified_at properties to the current date and time using new Date(). These operations ensure that the data is properly formatted and timestamps are added to track the creation and modification times of the request body.

  5. Fill in the form data and submit it. The data will be stored in your Azure Cosmos DB. The response will include the ID of the created record (document), similar to the following:

    {
  "id": "62f8ee487872a2325ceb3e73"
}
    {
  "id": "62f8ee487872a2325ceb3e73"
}

Displaying all the Stored Data

To display all the stored data in the database, you can follow these steps:

  1. Implement a new route handler in your index.js file as follows:

    /* Display all Bookings */
router.get('/booking', async function (req, res) {
    const db = await connectToDB();
    try {
        let results = await db.collection("bookings").find().toArray();
        res.render('bookings', { bookings: results });
    } catch (err) {
        res.status(400).json({ message: err.message });
    } finally {
        await db.client.close();
    }
});
    /* Display all Bookings */
router.get('/booking', async function (req, res) {
    const db = await connectToDB();
    try {
        let results = await db.collection("bookings").find().toArray();
        res.render('bookings', { bookings: results });
    } catch (err) {
        res.status(400).json({ message: err.message });
    } finally {
        await db.client.close();
    }
});

    This route handler retrieves all the data from the bookings collection and passes it to the bookings.ejs template for rendering.

  2. Create a new file named bookings.ejs in the views folder and add the following HTML code to it:

    <!DOCTYPE html>
<html>
  <head>
    <title>All Bookings</title>
    <link rel='stylesheet' href='/stylesheets/style.css' />
  </head>
  <body>
    <table>
        <% for (var booking of bookings) { %>
            <tr>
                <td><%= booking.email %></td>
                <td><%= booking.numTickets %></td>
            </tr>
        <% } %>
     </table>
  </body>
</html>
    <!DOCTYPE html>
<html>
  <head>
    <title>All Bookings</title>
    <link rel='stylesheet' href='/stylesheets/style.css' />
  </head>
  <body>
    <table>
        <% for (var booking of bookings) { %>
            <tr>
                <td><%= booking.email %></td>
                <td><%= booking.numTickets %></td>
            </tr>
        <% } %>
     </table>
  </body>
</html>

    This HTML file contains a table structure with placeholders for dynamic content using Embedded JavaScript (ejs) syntax. In the template, we loop over the bookings array and display the email and numTickets values for each booking.

  3. Visit http://localhost:3000/booking in your browser to see all the inputted data displayed in the table.

Zip your work and submit to our BUmoodle course room.

Appendix

Installation Guides

Microsoft Azure

01/02/2024 11:05